ACDS FELLOWSHIP: ASSURING STEM EDUCATION: DATA, CURRICULUM QUALITY ASSURANCE AND SCHOLARSHIP

Our claim We can learn a lot about how to assure the quality of curricula and delivery by reflecting on the characteristics of processes of change that worked well, as well as those that did not. Reflecting on processes of change also allows us to identify factors that we failed to consider in implementing change, and to identify strategies and processes that would have been useful to have in place. We believe that systematic and planned collection and analysis of data, as part of an ongoing analysis of Quality Improvement (QI) and Quality Assurance (QA) is a necessary component of risk mitigation strategies, that enables us to assure curriculum quality. This applies, in times of rapid change in response to sudden events, such as the response to COVID-19, but also throughout the normal cycle of curriculum design and implementation. Intended audience The workshop is relevant to educators who are interested in contributing to development of a sustainable and effective approach to educational evaluation and scholarship in STEM curriculum at the level of the degree. This includes teaching staff and/or teaching teams, degree, and unit coordinators, as well as Deans and Directors of Learning and Teaching. Learning outcomes By participating in this workshop, you will contribute to a round-table discussion that will build your knowledge of approaches and strategies to: • Enable and underscore the importance of collegial, scholarly activity directed toward learning and teaching. • Embed interacting cycles of QI, QA and Scholarship of Teaching and Learning (SoTL) processes into teaching practice and curriculum design at the level of a degree. • Develop capacity, teamwork, and leadership to integrate QA, QI and SoTL into the design, development, and delivery of curricula. • Identify resources needed to support a data-driven approach to quality curriculum design and delivery. • Identify ways to embed processes to gather natural data and deal with ethical issues into curriculum delivery and to use these data as part of an ongoing and continuous process of QI, QA and SoTL throughout the life of a course curriculum. Focus - main points This workshop focuses on the role of data in STEM education and on the value of planning and systematic implementation of evaluation and research as a key component of ongoing cycles of QI, QA and SoTL to ensure the best possible learning outcomes for students. Drawing on the unique strengths of STEM disciplines, we will consider strategies for: • embedding evidence-based, strategies into STEM education decision-making, curriculum design, development, and delivery • Identifying questions that focus scholarly attention on o quality improvement (issues and innovations), o quality assurance against standards and o student learning and engagement (impact and effectiveness of curriculum over time). • Identifying the types of data that should be routinely collected and analysed • Identifying strategies for data collection and analysis to ensure quality of curriculum and teaching. Time frame 2 hours, Including a 10 min break at the one-hour mark. Format Short presentation, structured collaborative discussion supported by breakout rooms and Google Docs shared document (one each breakout group; collated summary document). Method of interaction Using structured conversations in small groups (breakout rooms) and reporting back to whole group. Each breakout group will have a shared ‘google doc’ to record key points from their discussion, to be collated and shared with all participants post workshop

Responses to COVID-19 in the Context of Quality Assurance, Quality Improvement and Scholarship Requirements for Curricula

Our Curriculum Evaluation and Research-STEM Teaching Fellowship embeds leadership for active engagement in scholarship within teaching teams. It is a response to Higher Education Standards Framework (HESF) minimum requirements for continuous evaluation informing ongoing curriculum transformation, specifically the TEQSA Guidance Note: Scholarship (2018). The Fellowship contextualised the existing ‘Curriculum Evaluation Research (CER) framework’ (Kelder & Carr, 2017) for the specific characteristics of STEM degrees and teaching teams. The framework supports team-based planning and doing activities that are aligned with institutional structures, processes and governance instruments, so that scholarship can be made visible, monitored, measured, met and reported at the level of degree curriculum. Here we describe fellowship outcomes in the context of responses to COVID-19 using a case study at the University of Tasmania

Qualitative Research and Information Systems Design – Critical reflections from an eHealth Case study

Academic and business environments increasingly accept the utility of diverse qualitative research approaches for informing the design, implementation and evaluation of information systems. However, there are concerns that inherent techno-centrism within the IS discipline distorts criteria for choosing and adapting approaches and, significantly, works to marginalise the opportunities qualitative insights provide to open up human-centered dialogue on new ways of thinking and designing (Gasson 2003). This paper presents a qualitative research approach designed to facilitate critical reflection and sensitise researchers’ to implicit assumptions that technology will be the end-point of their activities and when judgments about criteria for successful designs are technologically and/or economically over-determined. The method endpoint is a conceptual framework constructed for the research domain which is the basis for translating sociological insights into implications for information systems and work practice in a public health service organisation

TALES FROM TASMANIA – SHORT- AND LONG-TERM RESPONSES TO COVID-19 IN THE CONTEXT OF QUALITY ASSURANCE, QUALITY IMPROVEMENT AND SCHOLARSHIP

“The COVID-19 response is a masterclass in the components of purposeful learning – and of using that learning to build an effective quality system – before our very eyes. Let us make them the foundation of our everyday learning and improvement tools now - and when we're talking about the COVID-19 days in the past tense.” Cathy Baldwin You can probably pin-point what you were doing, and where, when you first heard about COVID-19. It was my last day of annual leave when I started fielding calls from my Executive Dean about a novel coronavirus and how our students in China were effectively stranded. The next day, my first day back at the office, I recall chairing a meeting at 9 am to start planning our College response for transitioning affected units to online delivery. Roll forward another four weeks, this escalated to needing to shift as many units as feasibly possible to online delivery – which if you are in the higher education sector, you can relate to as a teacher, unit or degree coordinator, or in a leadership role. Working from home was the new normal, while being a few steps ahead of our students in preparing lectures and adapting practicals and rapidly gaining new web conferencing skills. It was at this time our university decided to accelerate a whole of institution academic transformation of our undergraduate degrees, ready for 2021, as part of a longer-term strategy towards a sustainable future. We were adapting our teaching delivery and planning for curricula change so quickly, it was at times a bit of a blur. For universities, as self-accrediting providers, it is imperative that our teaching and curricula are quality assured to meet legislative requirements. Even though that sounds a bit dry, as educators we surely want the best outcomes for our students. A fundamental question is how the online delivery and associated accommodations to assessment has impacted the quality of our teaching and learning. How will this be reflected in institutional and national quality indicators? How have the expectations of our students changed? It is a large and complicated task to draw this data together, and to have the approval to use it more broadly to communicate our successes and reflect on the challenges. What can we learn from our experiences dealing with COVID-19 to enable us to react to rapid change more effectively in the future? In mid-2019 (those were the days!) Jo Kelder and I were awarded the inaugural ACDS Teaching Fellowship. Initially proposed as a response to the Higher Education Standards Framework (HESF) minimum requirements that include continuous evaluation and the associated Guidance Note: Scholarship (TEQSA, 2018), our fellowship morphed to have a greater emphasis on quality assurance and quality improvement in the context of scholarship. These three interconnecting processes are conceptualised in a ‘Curriculum Evaluation Research (CER) framework’ for the specific characteristics of STEM degrees, with student learning at the core. Other outputs include national workshops at 11 institutions that introduce and refine the CER-STEM framework, a website, and a plan to share case studies and resources developed during the fellowship. We would love to be able to state that we had the framework implemented in our degrees – but with so many COVID-19 activities, this too had to be temporarily shelved. Reflecting on the disruption to curriculum content and delivery, we all know that some quality was necessarily sacrificed due to the short time frames - there was no opportunity to consult, no efficient mechanisms to monitor student learning experience and engagement. We believe it is important to think clearly about whether, and to what extent, the quality assurance planning embedded in the CER-STEM framework might have informed the context of rapidly evolving curricula. How mechanisms for formative and summative quality improvement could have been deployed to gather and analyse data for quality improvement more effectively. And how a developing culture of scholarly teaching practice might have ensured teaching teams were confident that change-decisions were based on evidence communicated through scholarship. Cathy Baldwin’s quote, above, reminds us of the need for an effective quality system, which we will explore in the context of the CER-framework. Using our College of Sciences and Engineering as an example, we plan to implement the framework across all our shiny new undergraduate degrees that will be offered from 2021 onwards. This approach will not only enable us to provide multi-faceted evidence of the impact of teaching delivery at the degree-level, but to demonstrate how and why our degrees offer a distinctively Tasmanian learning experience for our students. Not to mention that we will have a comprehensive evidence base with which to inform the next cycle of curriculum renewal

Social Constructionism with a Twist of Pragmatism: A Suitable Cocktail for Information Systems Research

The Information Systems discipline is concerned with the identification, planning, development, implementation and management of information systems in organisations. Thus, it is in organisations that the events, happenings and stories of information systems are played out. Given this, the perspective that Information Systems researchers have of organisational reality is most important. This viewpoint affects and impacts upon researchers’ practice, from the identification and choice of the research question, right through the analysis of data to the writing up of the research. This paper explores a particular theoretical perspective on organisational reality. Social constructionism augmented by pragmatism, is described and explored in the paper, and the implications for research Information Systems are considered. It is argued in the paper that this theoretical orientation is particularly helpful in planning, conceptualising and executing relevant and practical research in the Information Systems discipline

Preparing teachers for emergency remote teaching: A professional development framework for teachers in higher education

COVID-19 has significantly impacted teaching and learning in higher education, leading institutions to embrace Emergency Remote Teaching (ERT) in response to school and university closure. A systematic review research methodology was used to identify, analyse and synthesise literature on professional development in higher education published between 2010 and 2020. Following an inductive thematic analysis, the authors identified four themes that represent the literature: learning approaches, delivery modes, design features and institutional support. Based on the emerging themes and the analysis of the selection of studies, a framework for professional development is proposed to prepare teachers in higher education for ERT. The use of the framework is recommended to guide higher education institutions in best assisting their academic staff during an ERT context

ACDS Fellowship: Distributed leadership to embed scholarship in STEM teaching teams

In Australia, the Higher Education Standards Framework (HESF) minimum requirements include continuous evaluation that informs ongoing curriculum transformation: five threshold standards refer explicitly to scholarship. Significantly, TEQSA in December 2018 released the Guidance Note: Scholarship that states: ‘The intent of the Standards is that scholarship that is claimed to inform teaching (or supervision) must have a demonstrable relevance to the course being taught, including scholarship relating to the process of teaching and learning in itself.’ Changes to national Learning and Teaching awards and grants suggests a signal that scholarship is in practice an undervalued and largely invisible activity that may be neglected altogether by academics and management. In STEM, prioritisation of disciplinary research and lack of capacity/expertise in research approaches more common to the humanities and education are substantial barriers to scholarship in learning and teaching. Considering the TEQSA guidance note, it is essential to identify mechanisms to engage and reward academics to engage in scholarship of teaching and learning (SoTL). The HESF focus on degree-level curriculum implies the whole team of academics involved in designing and teaching degree curriculum need to be engaged in SoTL; not just specialist teachers. Our Fellowship will conceptualise leadership for active engagement in scholarship within teaching teams. It includes a Curriculum Evaluation Research (CER) framework (Kelder and Carr, 2017), which we have contextualised to the STEM discipline, which is designed to incorporate planning for SoTL into the practices of routine evidence collection that is response to institutional requirements for quality assurance of a body of curriculum and informed by national standards. The goal is to provide staff a practical and efficient method for ensuring coordinated quality activities related to a course that enable individual and collective outputs related to quality improvement, quality assurance and scholarship. Participants will be introduced to: 1. a way of thinking about the teaching component of academic work that is oriented to evidence-based quality enhancement 2. a framework for embedding evaluation and research into teaching practice, curriculum design and delivery at the level of a unit of study (subject) and course (award degree) curriculum that is aligned to the Higher Education Standards Framework and institutional performance expectations. 3. practical resources for data collection a. ‘generic’ ethics application; b. method for using the university Learning Management System (LMS) to routinely collect student and peer review data and; c. a mechanism for establishing consent. The workshop is relevant for teaching staff and/or teaching teams, course and unit coordinators, Deans and Directors of Learning and Teaching who are interested in establishing a sustainable and effective approach to educational evaluation and scholarship as part of their teaching practice. Associate Professor Tina Acuna and Dr Jo-Anne Kelder are the joint holders of the ACDS Fellowship for 2019. Tina is the Associate Dean Learning and Teaching in the College of Sciences and Engineering at UTAS. She led the national OLT funded AgLTAS project from 2013-15. Jo is Senior Lecturer in Curriculum Development and Innovation in the Tasmanian Institute of Learning and Teaching at UTAS. She led the OLT funded PATS extension grant and was a co-investigator in the AgLTAS project

Distributed leadership to embed scholarship in STEM teaching teams

In Australia, the Higher Education Standards Framework (HESF) minimum requirements include continuous evaluation that informs ongoing curriculum transformation. Significantly, TEQSA in 2018 released the Guidance Note: Scholarship and scholarship that is claimed to inform teaching must have a demonstrable relevance to the course being taught. Considering the TEQSA guidance note, it is essential to identify mechanisms to engage and reward academics to engage in SoTL. The HESF focus on degree-level curriculum implies the whole team of academics involved in designing and teaching degree curriculum need to be engaged in SoTL; not just specialist teachers. The value proposition for engagement in scholarship needs to resonate with academic goals and aspirations, rather than communicate administrative activities for compliance. Our joint ACDS Fellowship will conceptualise leadership for active engagement in scholarship within teaching teams. It includes a curriculum evaluation research (CER) framework, tailored for STEM, to ensure team-based scholarship activities are aligned with institutional structures, processes and instruments, so that scholarship can be focussed on degree-level curriculum and can be made visible, monitored, measured and reported against TEQSA standards

Learning and teaching academic standards for science: Where are we now?

One year on from the ALTC Learning and Teaching Academic Standards (LTAS) Project for Science, it is time to take stock of how the outcomes of that project are being implemented. In this paper, we shall discuss the current national regulatory environment, and what it might mean for us as practitioners in science education. We present examples of how the Science Threshold Learning Outcomes (TLOs) are being used in curriculum review and renewal and how they are being adapted to different disciplinary contexts